How Science Works 


How Science Works 

DAVID GOODSTEIN 

David Goodstein, B.S., M.S., Ph.D., is Vice Provost, Professor of Physics and Applied Physics, and the 
Frank J. Gilloon Distinguished Teaching and Service Professor, California Institute of Technology, Pasadena, 
California. 

CONTENTS 

I. A Bit of History, 2 

II. Theories of Science, 3 

A. Francis Bacons Scientific Method, 3 

B. Karl Popper s Falsification Theory, 4 

C. Thomas Kuhn’s Paradigm Shifts, 5 

D. An Evolved Theory of Science, 7 

III. Becoming a Professional Scientist, 9 

A. The Institutions, 9 

B. The Reward System and Authority Structure, 10 
IV Some Myths and Facts About Science, 11 

V. Comparing Science and the Law, 14 

A. Language, 14 

B. Objectives, 15 

VI. A Scientist’s View of Daubert, 15 


1 


1 


2/11/2000, 12:53 PM 



Reference Manual on Scientific Evidence 


R ecent Supreme Court decisions have put judges in the position of 
having to decide what is “scientific” and what is not. 1 Some judges 
may not feel entirely comfortable making such decisions, in spite of 
the guidance supplied by the Court and helpfully illuminated by learned com- 
mentators. 2 The purpose of this chapter is not to resolve the practical difficulties 
that judges will encounter in reaching those decisions, but, much more modestly, 
to demystify the business of science just a bit and to help judges understand the 
Daubert decision, at least as it appears to a scientist. In the hope of accomplishing 
these tasks, I take a mildly irreverent look at some formidable subjects. I hope the 
reader will accept this chapter in that spirit. 


I. A Bit of History 

Modern science can reasonably be said to have come into being during the time 
of Queen Elizabeth I of England and William Shakespeare. Almost immediately, 
it came into conflict with the law. 

While Shakespeare was composing his sonnets in England, Galileo Galilei in 
Italy was inventing the idea that careful experiments in a laboratory could reveal 
universal truths about the way objects move through space. A bit later, hearing 
about the newly invented telescope, he made one for himself and with it made 
discoveries in the heavens that astonished and thrilled all of Europe. Neverthe- 
less, in 1633, Galileo was put on trial for his scientific teachings. The trial of 
Galileo is usually portrayed as a conflict between science and the church, but it 
was, after all, a trial, with judges and lawyers, and all the other trappings of a 
formal legal procedure. 

Another great scientist of the day, William Harvey, who discovered the circu- 
lation of the blood, worked not only at the same time as Galileo, but even at the 
same place — the University of Padua, in Italy, not far from Venice. If one visits 
the University of Padua today and gets a tour of the old campus at the heart of 
the city, one will be shown Galileo’s cattedra, the wooden pulpit from which he 
lectured (and, curiously, one of his vertebrae in a display case just outside the 
rectors office — maybe the rector needs to be reminded to have a little spine). 


1 . These Supreme Court decisions are discussed in Margaret A. Berger, The Supreme Court’s Trilogy on 
the Admissibility of Expert Testimony, in this manual. For a discussion of the difficulty in distinguishing 
between science and engineering, see Henry Petroski, Reference Guide on Engineering Practice and Methods, 
in this manual. 

2. Since publication of the first edition of this manual, a number of works have been developed to 
assist judges and attorneys in understanding a wide range of scientific evidence. See, e.g, Modern Scientific 
Evidence: The Law and Science of Expert Testimony (David L. Faigman et al. eds., 1997); Expert Evi- 
dence: A Practitioners Guide to Law, Science, and the FJC Manual (Bert Black & Patrick W. Lee eds., 
1997). 
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One will also be shown the lecture-theater in which Harvey dissected cadavers 
while eager students peered downward from tiers of overhanging balconies. Dis- 
secting cadavers was illegal in Harvey’s time, so the floor of the theater was 
equipped with a mechanism to make the body disappear when a lookout gave 
the word that the authorities were coming. Of course, both science and the law 
have changed a great deal since the seventeenth century. 

Another important player who lived in the same era was not a scientist at all, 
but a lawyer who rose to be Lord Chancellor of England in the reign of James I, 
Elizabeth’s successor. His name was Sir Francis Bacon, and in his magnum opus, 
which he called Novum Organum, he put forth the first theory of the scientific 
method. In Bacon’s view, the scientist should be a disinterested observer of na- 
ture, collecting observations with a mind cleansed of harmful preconceptions 
that might cause error to creep into the scientific record. Once enough such 
observations have been gathered, patterns will emerge from them, giving rise to 
truths about nature. 

Bacon’s theory has been remarkably influential down through the ages, even 
though in his own time there were those who knew better. “That’s exactly how 
a Lord Chancellor would do science,”William Harvey is said to have grumbled. 


II. Theories of Science 

Today, in contrast to the seventeenth century, few would deny the central impor- 
tance of science to our lives, but not many would be able to give a good account 
of what science is. To most, the word probably brings to mind not science itself, 
but the fruits of science, the pervasive complex of technology that has trans- 
formed all of our lives. However, science might also be thought to include the 
vast body of knowledge we have accumulated about the natural world. There are 
still mysteries, and there always will be mysteries, but the fact is that, by and large, 
we understand how nature works. 

A. Francis Bacon’s Scientific Method 

But science is even more than that. If one asks a scientist the question, What is 
science?, the answer will almost surely be that science is a process, a way of 
examining the natural world and discovering important truths about it. In short, 
the essence of science is the scientific method . 3 


3. The Supreme Court, in Daubert v. Merrell Dow Pharmaceuticals, Inc., acknowledged the importance 
of defining science in terms of its methods as follows: “ ‘Science is not an encyclopedic body of knowl- 
edge about the universe. Instead, it represents a process for proposing and refining theoretical explanations 
about the world that are subject to further testing and refinement.’ ” (emphasis in original). 509 U.S. 579, 
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That stirring description suffers from an important shortcoming. We don’t 
really know what the scientific method is . 4 There have been many attempts at 
formulating a general theory of how science works, or at least how it ought to 
work, starting, as we have seen, with Sir Francis Bacon’s. Bacon’s idea, that sci- 
ence proceeds through the collection of observations without prejudice, has been 
rejected by all serious thinkers. Everything about the way we do science — the 
language we use, the instruments we use, the methods we use — depends on clear 
presuppositions about how the world works. Modern science is full of things 
that cannot be observed at all, such a force fields and complex molecules. At the 
most fundamental level, it is impossible to observe nature without having some 
reason to choose what is worth observing and what is not worth observing. 
Once one makes that elementary choice, Bacon has been left behind. 

B. Karl Popper’s Falsification Theory 

In this century, the ideas of the Austrian philosopher Sir Karl Popper have had a 
profound effect on theories of the scientific method . 5 In contrast to Bacon, 
Popper believed all science begins with a prejudice, or perhaps more politely, a 
theory or hypothesis. Nobody can say where the theory comes from. Formulat- 
ing the theory is the creative part of science, and it cannot be analyzed within 
the realm of philosophy. However, once the theory is in hand, Popper tells us, it 
is the duty of the scientist to extract from it logical but unexpected predictions 
that, if they are shown by experiment not to be correct, will serve to render the 
theory invalid. 

Popper was deeply influenced by the fact that a theory can never be proved 
right by agreement with observation, but it can be proved wrong by disagree- 
ment with observation. Because of this asymmetry, science makes progress uniquely 
by proving that good ideas are wrong so that they can be replaced by even better 
ideas. Thus, Bacon’s disinterested observer of nature is replaced by Popper’s skep- 
tical theorist. The good Popperian scientist somehow comes up with a hypoth- 
esis that fits all or most of the known facts, then proceeds to attack that hypoth- 
esis at its weakest point by extracting from it predictions that can be shown to be 
false. This process is known as falsification . 6 


590 (1993) (quoting Brief for the American Association for the Advancement of Science and the Na- 
tional Academy of Sciences as Amici Curiae at 7—8). 

4. For a general discussion of theories of the scientific method, see Alan F. Chalmers, What Is This 
Thing Called Science? (1982). For a discussion of the ethical implications of the various theories, see 
James Woodward & David Goodstein, Conduct, Misconduct and the Structure of Science, 84 Am. Scientist 
479, 479-90 (1996). 

5. See, e.g., Karl R. Popper, The Logic of Scientific Discovery (Karl R. Popper, trans., 1959). 

6. The Supreme Court in Daubert recognized Poppers conceptualization of scientific knowledge 
by noting that “[o]rdinarily, a key question to be answered in determining whether a theory or tech- 
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Popper’s ideas have been fruitful in weaning the philosophy of science away 
from the Baconian view and some other earlier theories, but they fall short in a 
number of ways in describing correctly how science works. The first of these is 
the observation that, although it may be impossible to prove a theory is true by 
observation or experiment, it is nearly just as impossible to prove one is false by 
these same methods. Almost without exception, in order to extract a falsifiable 
prediction from a theory, it is necessary to make additional assumptions beyond 
the theory itself. Then, when the prediction turns out to be false, it may well be 
one of the other assumptions, rather than the theory itself, that is false. To take a 
simple example, early in the twentieth century it was found that the orbits of the 
outermost planets did not quite obey the predictions of Newton’s laws of gravity 
and mechanics. Rather than take this to be a falsification of Newton’s laws, 
astronomers concluded the orbits were being perturbed by an additional unseen 
body out there. They were right. That is precisely how the planet Pluto was 
discovered. 

The apparent asymmetry between falsification and verification that lies at the 
heart of Popper’s theory thus vanishes. But the difficulties with Popper’s view go 
even beyond that problem. It takes a great deal of hard work to come up with a 
new theory that is consistent with nearly everything that is known in any area of 
science. Popper’s notion that the scientist’s duty is then to attack that theory at its 
most vulnerable point is fundamentally inconsistent with human nature. It would 
be impossible to invest the enormous amount of time and energy necessary to 
develop a new theory in any part of modern science if the primary purpose of all 
that work was to show that the theory was wrong. 

This point is underlined by the fact that the behavior of the scientific com- 
munity is not consistent with Popper’s notion of how it should be. Credit in 
science is most often given for offering correct theories, not wrong ones, or for 
demonstrating the correctness of unexpected predictions, not for falsifying them. 
I know of no example of a Nobel Prize awarded to a scientist for falsifying his or 
her own theory. 

C. Thomas Kuhn’s Paradigm Shifts 

Another towering figure in the twentieth century theory of science is Thomas 
Kuhn . 7 Kuhn was not a philosopher but a historian (more accurately, a physicist 


nique is scientific knowledge that will assist the trier of fact will be whether it can be (and has been) 
tested.” 509 U.S. at 593. In support of this point, the Court cited as parentheticals passages from both 
Carl Gustav Hempel, Philosophy of Natural Science 49 (1966) (“ ‘[T]he statements constituting a scientific 
explanation must be capable of empirical test’ ”), and Karl R. Popper, Conjectures and Refutations: The 
Growth of Scientific Knowledge 37 (5th ed. 1989) (“ ‘[T]he criterion of the scientific status of a theory 
is its falsifiability, or refutability, or testability’ ”). 

7. Thomas S. Kuhn, The Structure of Scientific Revolutions (1962). 
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who retrained himself as a historian). It is Kuhn who popularized the word 
paradigm, which has today come to seem so inescapable. 

A paradigm, for Kuhn, is a sort of consensual world view within which scien- 
tists work. It comprises an agreed upon set of assumptions, methods, language, 
and everything else needed to do science. Within a given paradigm, scientists 
make steady, incremental progress, doing what Kuhn calls “normal science.” 

As time goes on, difficulties and contradictions arise that cannot be resolved, 
but one way or another, they are swept under the rug, rather than being allowed 
to threaten the central paradigm. However, at a certain point, enough of these 
difficulties have accumulated so that the situation becomes intolerable. At that 
point, a scientific revolution occurs, shattering the paradigm and replacing it 
with an entirely new one. 

The new paradigm is so radically different from the old that normal discourse 
between the practitioners of the two paradigms becomes impossible. They view 
the world in different ways and speak different languages. It isn’t even possible to 
tell which of the two paradigms is superior, because they address different sets of 
problems. They are incommensurate. Thus, science does not progress incremen- 
tally, as the science textbooks would have it, except during periods of normal 
science. Every once in a while, a scientific revolution brings about a paradigm 
shift, and science heads off in an entirely new direction. 

Kuhn’s view was formed largely on the basis of two important historical revo- 
lutions. One was the original scientific revolution that started with Nicolaus 
Copernicus and culminated with the new mechanics of Isaac Newton. The very 
word revolution, whether it refers to the scientific kind, the political kind, or any 
other kind, refers metaphorically to the revolutions in the heavens that Copernicus 
described in a book, De Revolutionibus Orbium Caelestium, which was published 
as he lay dying in 1543. 8 Before Copernicus, the dominant paradigm was the 
world view of ancient Greek philosophy, frozen in the fourth century B.C. ideas 
of Plato and Aristotle. After Newton, whose masterwork, Philosophies Naturalis 
Principia Mathematica, was published in 1687, every scientist was a Newtonian, 
and Aristotelianism was banished forever from the world stage. It is even possible 
that Sir Francis Bacon’s disinterested observer was a reaction to Aristotelian au- 
thority. Look to nature, not to the ancient texts, Bacon may have been saying. 

The second revolution that served as an example for Kuhn occurred early in 
the twentieth century. In a headlong series of events that lasted a mere twenty- 
five years, the Newtonian paradigm was overturned and replaced with the new 
physics, in the form of quantum mechanics and Einstein’s relativity. The second 
revolution, though it happened much faster, was no less profound than the first. 

The idea that science proceeds by periods of normal activity punctuated by 
shattering breakthroughs that make scientists rethink the whole problem is an 

8. I. Bernard Cohen, Revolution in Science (1985). 


6 


How Science Works 


2/11/2000, 12:53 PM 



Hou > Science Works 


appealing one, especially to the scientists themselves, who know from personal 
experience that it really happens that way. Kuhns contribution is important. It 
gives us a new and useful structure (a paradigm, one might say) for organizing 
the entire history of science. 

Nevertheless, Kuhn’s theory does suffer from a number of shortcomings as an 
explanation for how science works. One of them is that it contains no measure 
of how big the change must be in order to count as a revolution or paradigm 
shift. Most scientists will say that there is a paradigm shift in their laboratory 
every six months or so (or at least every time it becomes necessary to write 
another proposal for research support) .That isn’t exactly what Kuhn had in mind. 

Another difficulty is that even when a paradigm shift is truly profound, the 
paradigms it separates are not necessarily incommensurate. The new sciences of 
quantum mechanics and relativity, for example, did indeed show that Newton’s 
laws of mechanics were not the most fundamental laws of nature. However, they 
did not show that they were wrong. Quite the contrary, they showed why 
Newton’s laws of mechanics were right: Newton’s laws arose out of new laws 
that were even deeper and that covered a wider range of circumstances unimagined 
by Newton and his followers, that is, things as small as atoms, or nearly as fast as 
the speed of light, or as dense as black holes. In more familiar realms of experi- 
ence, Newton’s laws go on working just as well as they always did. Thus, there is 
no ambiguity at all about which paradigm is better. The new laws of quantum 
mechanics and relativity subsume and enhance the older Newtonian world. 

D. An Evolved Theory of Science 

If neither Bacon nor Popper nor Kuhn gives us a perfect description of what 
science is or how it works, nevertheless all three help us to gain a much deeper 
understanding of it all. 

Scientists are not Baconian observers of nature, but all scientists become 
Baconians when it comes to describing their observations. Scientists are rigor- 
ously, even passionately honest about reporting scientific results and how they 
were obtained, in formal publications. Scientific data are the coin of the realm in 
science, and they are always treated with reverence. Those rare instances in which 
data are found to have been fabricated or altered in some way are always trau- 
matic scandals of the first order . 9 

Scientists are also not Popperian falsifiers of their own theories, but they don’t 
have to be. They don’t work in isolation. If a scientist has a rival with a different 

9. Such instances are discussed in David Goodstein, Scientific Fraud, 60 Am. Scholar 505 (1991). For 
a summary of recent investigations into scientific fraud and lesser instances of scientific misconduct, see 
Office of Research Integrity, Department of Health and Human Services, Scientific Misconduct Inves- 
tigations: 1993—1997 <http://ori.dhhs.gov/PDF/scientific.pdf> (summarizing 150 scientific miscon- 
duct investigations closed by the Office of Research Integrity). 
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theory of the same phenomena, the rival will be more than happy to perform 
the Popperian duty of attacking the scientist’s theory at its weakest point. More- 
over, if falsification is no more definitive than verification, and scientists prefer in 
any case to be right rather than wrong, they nevertheless know how to hold 
verification to a very high standard. If a theory makes novel and unexpected 
predictions, and those predictions are verified by experiments that reveal new 
and useful or interesting phenomena, then the chances that the theory is correct 
are greatly enhanced. And even if it is not correct, it has been fruitful in the sense 
that it has led to the discovery of previously unknown phenomena that might 
prove useful in themselves and that will have to be explained by the next theory 
that comes along. 

Finally, science does not, as Kuhn seemed to think, periodically self-destruct 
and need to start over again, but it does undergo startling changes of perspective 
that lead to new and, invariably, better ways of understanding the world. Thus, 
science does not proceed smoothly and incrementally, but it is one of the few 
areas of human endeavor that is truly progressive. There is no doubt at all that 
twentieth century science is better than nineteenth century science, and we can 
be absolutely confident that what will come along in the twenty-first century 
will be better still. One cannot say the same about, say, art or literature . 10 

To all this, a couple of things must be added. The first is that science is, above 
all, an adversary process. It is a arena in which ideas do battle, with observations 
and data the tools of combat. The scientific debate is very different from what 
happens in a court of law, but just as in the law, it is crucial that every idea receive 
the most vigorous possible advocacy, just in case it might be right. Thus, the 
Popperian ideal of holding one’s hypothesis in a skeptical and tentative way is 
not merely inconsistent with reality, it would be harmful to science if it were 
pursued. As I discuss shortly, not only ideas, but the scientists themselves engage 
in endless competition according to rules that, although they are nowhere writ- 
ten down, are nevertheless complex and binding. 


10. The law, too, can claim to be progressive. Development of legal constructs, such as due process, 
equal protection, and individual privacy, reflects notable progress in the betterment of mankind. See 
Laura Kalman, The Strange Career of Legal Liberalism 2—4 (1996) (recognizing the “faith” of legal 
liberalists in the use of law as an engine for progressive social change in favor of society’s disadvantaged). 
Such progress is measured by a less precise form of social judgment than the consensus that develops 
regarding scientific progress. See Steven Goldberg, The Reluctant Embrace: Law and Science in America, 75 
Geo. L.J. 1341, 1346 (1987) (“Social judgments, however imprecise, can sometimes be reached on legal 
outcomes. If a court s decision appears to lead to a sudden surge in the crime rate, it may be judged 
wrong. If it appears to lead to new opportunities for millions of citizens, it may be judged right. The law 
does gradually change to reflect this kind of social testing. But the process is slow, uncertain, and contro- 
versial; there is nothing in the legal community like the consensus in the scientific community on 
whether a particular result constitutes progress”) 
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In the competition among ideas, the institution of peer review plays a central 
role. Scientific articles submitted for publication and proposals for funding are 
often sent to anonymous experts in the field, in other words, peers of the author, 
for review. Peer review works superbly to separate valid science from nonsense, 
or, in Kuhnian terms, to ensure that the current paradigm has been respected . 1 1 It 
works less well as a means of choosing between competing valid ideas, in part 
because the peer doing the reviewing is often a competitor for the same re- 
sources (pages in prestigious journals, funds from government agencies) being 
sought by the authors. It works very poorly in catching cheating or fraud, be- 
cause all scientists are socialized to believe that even their bitterest competitor is 
rigorously honest in the reporting of scientific results, making it easy to fool a 
referee with purposeful dishonesty if one wants to. Despite all of this, peer re- 
view is one of the sacred pillars of the scientific edifice. 


III. Becoming a Professional Scientist 

Science as a profession or career has become highly organized and structured . 12 
It is not, relatively speaking, a very remunerative profession — that would be in- 
consistent with the Baconian ideal — but it is intensely competitive, and a certain 
material well-being does tend to follow in the wake of success (successful scien- 
tists, one might say, do get to bring home the Bacon). 

A. The Institutions 

These are the institutions of science: Research is done in the Ph.D. -granting 
universities, and to a lesser extent, in colleges that don’t grant Ph.D.s. It is also 
done in national laboratories and in industrial laboratories. Before World War II, 
basic science was financed mostly by private foundations (Rockefeller, Carnegie), 
but since the war, the funding of science (except in industrial laboratories) has 

1 1 . The Supreme Court received differing views regarding the proper role of peer review. Compare 
Brief for Amici Curiae Daryl E. Chubin et al. at 10, Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579 
(1993) (No. 92-102) (“peer review referees and editors limit their assessment of submitted articles to 
such matters as style, plausibility, and defensibility; they do not duplicate experiments from scratch or 
plow through reams of computer-generated data in order to guarantee accuracy or veracity or cer- 
tainty”) with Brief for Amici Curiae New England Journal of Medicine, Journal of the American Medi- 
cal Association, and Annals of Internal Medicine in Support of Respondent, Daubert v. Merrell Dow 
Pharms., Inc., 509 U.S. 579 (1993) (No. 92-102) (proposing that publication in a peer-reviewed journal 
be the primary criterion for admitting scientific evidence in the courtroom). See generally Daryl E. 
Chubin & Edward J. Hackett, Peerless Science: Peer Review and U.S. Science Policy (1990); Arnold S. 
Reiman & Marcia Angell, How Good Is Peer Review? 321 New Eng. J. Med. 827—29 (1989). As a practic- 
ing scientist and frequent peer reviewer, I can testify that Chubin’s view is correct. 

12. The analysis that follows is based on David Goodstein & James Woodward, Inside Science, Ameri- 
can Scholar (forthcoming). 
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largely been taken over by agencies of the federal government, notably the Na- 
tional Science Foundation (an independent agency), the National Institutes of 
Health (part of the Public Health Service of the Department of Health and 
Human Services), and parts of the Department of Energy and the Department 
of Defense. 

Scientists who work at all these organizations — universities, colleges, national 
and industrial laboratories, and funding agencies — belong to scientific societies 
that are organized mostly by discipline. There are large societies, such as the 
American Physical Society and the American Chemical Society; societies for 
subdisciplines, such as optics and spectroscopy; and even organizations of societ- 
ies, such as faseb (the Federation of American Societies of Experimental Biol- 
ogy)- 

Scientific societies are private organizations that elect their own officers, hold 
scientific meetings, publish journals, and finance their operations from the col- 
lection of dues and from the proceeds of their publishing and educational activi- 
ties. The American Association for the Advancement of Science also holds meet- 
ings and publishes a famous journal (Science), but it is not restricted to any one 
discipline. The National Academy of Sciences holds meetings and publishes a 
journal, and it has an operational arm, the National Research Council, that car- 
ries out studies for various government agencies, but by far its most important 
activity is to elect its own members. 

These are the basic institutions of American science. It should not come as 
news that the universities and colleges engage in a fierce but curious competi- 
tion, in which no one knows who’s keeping score, but everyone knows roughly 
what the score is. (In recent years, some national newsmagazines have found it 
profitable to appoint themselves scorekeepers in this competition. Academic 
officials dismiss these journalistic judgments, except when their own institutions 
come out on top.) Departments in each discipline compete with one another, as 
do national and industrial laboratories and even funding agencies. Competition 
in science is at its most refined, however, at the level of individual careers. 

B. The Reward System and Authority Structure 

To regulate the competition among scientists, there is a reward system and an 
authority structure. The fruits of the reward system are fame, glory, and immor- 
tality. The purposes of the authority structure are power and influence. The re- 
ward system and the authority structure are closely related to one another, but 
scientists distinguish sharply between them. When they speak of a colleague who 
has become president of a famous university, they will say sadly, “It’s a pity — he 
was still capable of good work,” sounding like warriors lamenting the loss of a 
fallen comrade. The university president is a kingpin of the authority structure, 
but he is a dropout from the reward system. Similar sorts of behavior can be 
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observed in industrial and government laboratories, but a description of what 
goes on in universities will be enough to illustrate how the system works. 

A career in academic science begins at the first step on the reward system 
ladder, a Ph.D., followed in many areas by one or two stints as a postdoctoral 
fellow.The Ph.D. and postdoctoral positions had best be at universities (or at least 
departments) that are high up in that fierce but invisible competition because all 
subsequent steps are most likely to take the individual sideways or downward on 
the list. The next step is a crucial one: appointment to a tenure-track junior 
faculty position. About two-thirds of all postdoctoral fellows in American uni- 
versities believe they are going to make this step, but in fact, only about a quarter 
of them succeed. This step and all subsequent steps require growing fame as a 
scientist beyond the individual’s own circle of acquaintances. Recommendations 
will be sought from people who know of the person because of the importance 
of his or her scientific accomplishments. Thus, it is essential by this time that the 
individual has accomplished something. The remaining steps up the reward sys- 
tem ladder are promotion to an academic tenured position and full professorship; 
various prizes, medals, and awards given out by the scientific societies; an en- 
dowed chair (the virtual equivalent of Galileo’s wooden cattedra ); election to the 
National Academy; the Nobel Prize; and, finally, immortality. 

Positions in the authority structure are generally rewards for having achieved 
a certain level in the reward system. For example, starting from the junior faculty 
level, it is possible to step sideways temporarily or even permanently into a posi- 
tion as contract officer in a funding agency. Because contract officers influence 
the distribution of research funds, they have a role in deciding who will succeed 
in the climb up the reward system ladder. At successively higher levels one can 
become the editor of a journal; chair of a department; dean, provost, or president 
of a university; and even the head of a funding agency. People in these positions 
have stepped out of the reward system, but they have something to say about 
who succeeds in it. 


IV Some Myths and Facts About Science 

“In matters of science,” Galileo wrote, “the authority of thousands is not worth 
the humble reasoning of one single person .” 13 Doing battle with the Aristotelian 
professors of his day, Galileo believed that appeal to authority was the enemy of 


13. I found this statement framed on the office wall of a colleague in Italy in the form “In questioni 
di scienza L’autorita di mille non vale I’umile ragionare di un singolo.” However, I have not been able to find 
the famous remark in this form in Galileo’s writings. An equivalent statement in different words can be 
found in Galileos II Saggiatore (1623). See Andrea Frova & Mariapiera Marenzona, Parola di Galileo 473 
(1998). 
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reason. But, contrary to Galileo’s famous remark, the fact is that authority is of 
fundamental importance to science. If a paper’s author is a famous scientist, I 
think the paper is probably worth reading. However, an appeal from a scientific 
wanna-be, asking that his great new discovery be brought to the attention of the 
scientific world, is almost surely not worth reading (such papers arrive in my 
office, on the average, about once a week). The triumph of reason over authority 
is just one of the many myths about science, some of which I’ve already dis- 
cussed. Here’s a brief list of others: 

Myth: Scientists must have open minds, being ready to discard old ideas in favor 
of new ones. 

Fact: Because science is an adversary process in which each idea deserves the 
most vigorous possible defense, it is useful for the successful progress of sci- 
ence that scientists tenaciously hang on to their own ideas, even in the face of 
contrary evidence (and they do, they do). 

Myth: Science must be an open book. For example, every new experiment must 
be described so completely that any other scientist can reproduce it. 

Fact: There is a very large component of skill in making cutting-edge experi- 
ments work. Often, the only way to import a new technique into a laboratory 
is to hire someone (usually a postdoctoral fellow) who has already made it 
work elsewhere. Nevertheless, scientists have a solemn responsibility to de- 
scribe the methods they use as fully and accurately as possible. And, eventually, 
the skill will be acquired by enough people to make the new technique com- 
monplace. 

Myth: When a new theory comes along, the scientist’s duty is to falsify it. 

Fact: When a new theory comes along, the scientist’s instinct is to verify it. When 
a theory is new, the effect of a decisive experiment that shows it to be wrong 
is that both the theory and the experiment are quickly forgotten. This result 
leads to no progress for anyone in the reward system. Only when a theory is 
well established and widely accepted does it pay off to prove that it’s wrong. 

Myth: Real science is easily distinguished from pseudoscience. 

Fact: This is what philosophers call the problem of demarcation: One of Popper’s 
principal motives in proposing his standard of falsifiability was precisely to 
provide a means of demarcation between real science and impostors. For 
example, Einstein’s theory of relativity (with which Popper was deeply im- 
pressed) made clear predictions that could certainly be falsified if they were 
not correct. In contrast, Freud’s theories of psychoanalysis (with which Pop- 
per was far less impressed) could never be proven wrong. Thus, to Popper, 
relativity was science but psychoanalysis was not. 

As I’ve already shown, real scientists don’t do as Popper says they should. 
But quite aside from that, there is another problem with Popper’s criterion 
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(or indeed any other criterion) for demarcation: would-be scientists read books, 
too. If it becomes widely accepted (and to some extent it has) that falsifiable 
predictions are the signature of real science, then pretenders to the throne of 
science will make falsifiable predictions, too . 14 There is no simple, mechanical 
criterion for distinguishing real science from something that is not real sci- 
ence. That certainly doesn’t mean, however, that the job can’t be done. As I 
discuss below, the Supreme Court, in the Daubert decision, has made a re- 
spectable stab at showing how to do it . 15 

Myth: Scientific theories are just that: theories. All scientific theories are eventu- 
ally proved wrong and are replaced by other theories. 

Fact: The things that science has taught us about how the world works are the 
most secure elements in all of human knowledge. I must distinguish here 
between science at the frontiers of knowledge (where by definition we don’t 
yet understand everything and where theories are indeed vulnerable) and 
textbook science that is known with great confidence. Matter is made of 
atoms, dna transmits the blueprints of organisms from generation to genera- 
tion, light is an electromagnetic wave; these things are not likely to be proved 
wrong. The theory of relativity and the theory of evolution are in the same 
class. They are still called theories for historic reasons only. The satellite navi- 
gation system in my car routinely uses the theory of relativity to make calcu- 
lations accurate enough to tell me exactly where I am and to take me to my 
destination with unerring precision. 

It should be said here that the incorrect notion that all theories must even- 
tually be wrong is fundamental to the work of both Popper and Kuhn, and 
these theorists have been crucial in helping us understand how science works. 
Thus, their theories, like good scientific theories at the frontiers of knowl- 
edge, can be both useful and wrong. 

Myth: Scientists are people of uncompromising honesty and integrity. 

Fact: They would have to be if Bacon were right about how science works, but 
he wasn’t. Scientists are rigorously honest where honesty matters most to 


14. For a list of such pretenders, see Larry Landan, Beyond Positivism and Relativism 219 (1996). 

15. The Supreme Court in Daubert identified four nondefinitive factors that were thought to be 
illustrative of characteristics of scientific knowledge: testability or falsifiability, peer review, a known or 
potential error rate, and general acceptance within the scientific community. 509 U.S. at 590 (1993). 
Subsequent cases have expanded on these factors. See, eg, In re TMI Litig. Cases Consol. II, 91 1 F. Supp. 
775, 787 (M.D. Pa. 1995) (which considered the following additional factors: the relationship of the 
technique to methods that have been established to be reliable; the qualifications of the expert witness 
testifying based on the methodology; the nonjudicial uses of the method; logical or internal consistency 
of the hypothesis; consistency of the hypothesis with accepted theories; and precision of the hypothesis 
or theory). See generally Bert Black et al., Science and the Law in the Wake of Daubert: A New Search for 
Scientific Knowledge, 72 Tex. L. Rev. 715, 783—84 (1994) (discussion of expanded list of factors). 
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them: in the reporting of scientific procedures and data in peer-reviewed 
publications. In all else, they are ordinary mortals like all other ordinary mor- 
tals. 


V. Comparing Science and the Law 

Science and the law differ in both the language they use and the objectives they 
seek to accomplish. 

A. Language 

Someone once said that the United States and England are two nations separated 
by a common language. Something similar can be said of science and the law. 
There are any number of words that are commonly used in both disciplines, but 
with different meanings. Let me give just a few examples. 

The word force, as it is used by lawyers, has connotations of violence and the 
domination of one person’s will over another, as in phrases such as “excessive use 
of force” and “forced entry.” In science, force is something that when applied to 
a body, causes its speed and direction of motion to change. Also, all forces arise 
from a few fundamental forces, most notably gravity and the electric force. The 
word carries no other baggage. 

In contrast, the word evidence is used much more loosely in science than in the 
law. The law has precise rules of evidence that govern what is admissible and 
what isn’t. In science the word merely seems to mean something less than “proof.” 
A certain number of the papers in any issue of a scientific journal will have titles 
that begin with “Evidence for (or against) .” What that means is, the authors 
weren’t able to prove their point, but here are their results anyway. 

The word theory is a particularly interesting example of a word that has differ- 
ent meanings in the two disciplines. A legal theory (as I understand it) is a pro- 
posal that fits the known facts and legal precedents and that favors the attorney’s 
client. The requisite of a theory in science is that it make new predictions that 
can be tested by new experiments or observations and falsified or verified (as 
discussed above), but in any case, put to the test. 

Even the word law has different meanings in the two disciplines. To a legal 
practitioner, a law is something that has been promulgated by some human au- 
thority, such as a legislature or parliament. In science, a law is a law of nature, 
something that humans can hope to discover and describe accurately, but that 
can never be changed by any human authority. 

My final example is, to me, the most interesting of all. It is the word error. In 
the law, and in common usage, error and mistake are more or less synonymous. A 
legal decision can be overturned if it is found to be contaminated by judicial 
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error. In science, however, error and mistake have different meanings. Anyone can 
make a mistake, and scientists have no obligation to report theirs in the scientific 
literature. They just clean up the mess and go on to the next attempt. Error, on 
the other hand, is intrinsic to any measurement, and far from ignoring it or 
covering it up or even attempting to eliminate it, authors of every paper about a 
scientific experiment will include a careful analysis of the errors to put limits on 
the uncertainty in the measured result. To make mistakes is human, one might 
say, but error is intrinsic to our interaction with nature, and is therefore part of 
science. 

B. Objectives 

Beyond the meanings of certain key words, science and the law differ fundamen- 
tally in their objectives. The objective of the law is justice; that of science is 
truth. 16 These are not at all the same thing. Justice, of course, also seeks truth, but 
it requires that a clear decision be made in a reasonable and limited amount of 
time. In the scientific search for truth there are no time limits and no point at 
which a final decision must be made. 

And yet, in spite of all these differences, science and the law share, at the 
deepest possible level, the same aspirations and many of the same methods. Both 
disciplines seek, in structured debate, using empirical evidence, to arrive at ratio- 
nal conclusions that transcend the prejudices and self-interest of individuals. 


VI. A Scientist’s View of Daubert 

In the 1993 Daubert decision, the U.S. Supreme Court took it upon itself to 
solve, once and for all, the knotty problem of the demarcation of science from 
pseudoscience. Better yet, it undertook to enable every federal judge to solve 
that problem in deciding the admissibility of each scientific expert witness in 
every case that arises. In light of all the uncertainties discussed in this chapter, it 
must be considered an ambitious thing to do. 17 


16. This point is made eloquently by D. Allen Bromley in Science and the Law, Address at the 1998 
Annual Meeting of the American Bar Association (Aug. 2, 1998). 

17. Chief Justice Rehnquist, responding to the majority opinion in Daubert, was the first to express 
his uneasiness with the task assigned to federal judges as follows: “I defer to no one in my confidence in 
federal judges; but I am at a loss to know what is meant when it is said that the scientific status of a theory 
depends on its ‘falsifiability,’ and I suspect some of them will be, too.” 509 U.S. 579, 600 (1993) (Rehnquist, 
C.J., concurring in part and dissenting in part). His concern was then echoed by Judge Alex Kozinski 
when the case was reconsidered by the U.S. Court of Appeals for the Ninth Circuit following remand by 
the Supreme Court. 43 F.3d 1311, 1316 (9th Cir. 1995) (“Our responsibility, then, unless we badly 
misread the Supreme Court s opinion, is to resolve disputes among respected, well-credentialed scien- 
tists about matters squarely within their expertise, in areas where there is no scientific consensus as to 
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The presentation of scientific evidence in a court of law is a kind of shotgun 
marriage between the two disciplines. Both are forced to some extent to yield to 
the central imperatives of the other’s way of doing business, and it is likely that 
neither will be shown in its best light. The Daubert decision is an attempt (not the 
first, of course) to regulate that encounter. Judges are asked to decide the “evi- 
dential reliability” of the intended testimony, based not on the conclusions to be 
offered, but on the methods used to reach those conclusions. 

In particular, the methods should be judged by the following four criteria: 

1 . The theoretical underpinnings of the methods must yield testable predic- 
tions by means of which the theory could be falsified. 

2. The methods should preferably be published in a peer-reviewed journal. 

3. There should be a known rate of error that can be used in evaluating the 
results. 

4. The methods should be generally accepted within the relevant scientific 
community. 

In reading these four illustrative criteria mentioned by the Court, one is struck 
immediately by the specter of Karl Popper looming above the robed justices. (It’s 
no mere illusion. The dependence on Popper is explicit in the written decision.) 
Popper alone is not enough, however, and the doctrine of falsification is supple- 
mented by a bow to the institution of peer review, an acknowledgment of the 
scientific meaning of error, and a paradigm check (really, an inclusion of the 
earlier Frye standard). 18 

All in all, I would score the decision a pretty good performance. 19 The justices 
ventured into the treacherous crosscurrents of the philosophy of science — where 
even most scientists fear to tread — and emerged with at least their dignity intact. 
Falsifiability may not be a good way of doing science, but it’s not the worst a 
posteriori way to judge science, and that’s all that’s required here. At least they 
managed to avoid the Popperian trap of demanding that the scientists be skepti- 
cal of their own ideas .The other considerations help lend substance and flexibility. 20 
The jury is still out (so to speak) on how well this decision will work in practice, 
but it’s certainly an impressive attempt to serve justice, if not truth. Applying it in 
practice will never be easy, but then that’s what this manual is all about. 


what is and what is not ‘good science,’ and occasionally to reject such expert testimony because it was 
not ‘derived by the scientific method.’ Mindful of our position in the hierarchy of the federal judiciary, 
we take a deep breath and proceed with this heady task.”) 

18. In Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923), the court stated that expert opinion 
based on a scientific technique is inadmissible unless the technique is “generally accepted” as reliable in 
the relevant scientific community. 

19. For a contrary view, see Gary Edmond & David Mercer, Recognizing Daubert: What Judges 
Should Know About Falsification, 5 Expert Evidence 29—42 (1996). 

20. See supra note 15. 
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